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DETAILED ACTION 

1 . The Office withdraws the previous rejections of the claims under 35 USC § 103(a), in 
light of the amendment. However, the Office sets forth new rejections of the claims under 35 
USC §§1 12-2 nd paragraph and 103(a), in light of the amendment. 

Response to Arguments 

2. Applicant's arguments with respect to claims have been considered but are moot in view 
of the new ground(s) of rejection. 

Continued Examination Under 37 CFR 1.114 

3 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1 . 17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1 . 17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 10/30/2007 has been entered. 

Claim Rejections - 35 USC § 112 

4. The following is a quotation of the second paragraph of 35 U.S. C. 1 12: 



The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 
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5. Claim 31 is rejected under 35 U.S.C 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. This claim is vague and ambiguous, and thus, its scope is indeterminable. 

Regarding claim 31: This claim recites "a timeline". It is unclear whether this term is 
referring to the same "timeline" recited in parent claim 25 in lines 8 and 12. Thus, this claim is 
vague and ambiguous, and its scope is indeterminable. 

Claim Rejections - 35 USC §103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7 Claims 1-6, 8, 10, 12-22, 25, 27, 31, 33-42 and 44-45 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Scott R. Klemmer et al. ("Books With Voices: Paper 
Transcripts as a Tangible Interface to Oral Histories", CHI 2003 . Fort Lauderdale, FL, Apr. 5-10, 
2003, pp. 89-96, hereafter referred to as "Klemmer") in view of Graham et al (US Patent No. 
6,369,81 1, filed Sep. 9, 1998 and issued Apr. 9, 2002, hereafter referred to as "Graham"). 
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Regarding independent claim 1 : Klemmer teaches A computer system for generating 
a representation of time-based media, the system comprising: a feature extraction module for: 
extracting features from media content; (See Klemmer page 92 in the top paragraph of the right 
column discussing the creating of an MPEG-2 video from a video source.) and generating a 
media representation representing the features extracted; (See Klemmer page 92 in the top 
paragraph of the right column discussing the making of corresponding JPEG thumbnails.) a 
formatting module for formatting the media representation generated, the formatting module 
being communicatively coupled to the feature extraction module to apply features extracted to 
the media representation, wherein the formatting module formats the media representation 
according to a representation specification; (See Klemmer page 92 in the top paragraph of the 
right column discussing the creating of a paper layout from a time stamped transcript.) wherein 
the formatted media representation includes a graphical representation of a timeline and a 
plurality of user selectable identifiers representing the features extracted from the media 
content for selection by a user to play media content segments of a defined length associated 
with each of the features, wherein the plurality of selectable identifiers are linked to locations 
on the timeline. (See Klemmer page 91 in the last paragraph of the right column discussing the 
adding of time code information to the print format. See also page 92 Fig. 3 showing barcodes 
linking chronological sections of a book to A/V data and page 92 in the top paragraph of the 
right column discussing the creating of a paper layout from a time stamped transcript.) 

However, Klemmer does not explicitly teach the remaining limitations as claimed. 
Graham, though, discloses and a printer for printing the formatted media representation, the 
printer being communicatively coupled to the formatting module to receive instructions for 
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printing a document displaying the formatted media representation, (See Graham Fig. 4A 
#408 and col. 5 lines 32-50 teaching the printing of a paper reader's assistant document having 
an imprinted thumbnail and teaching additional information as being a "discussion" of user 
interests.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Graham for the benefit of Klemmer, because to do so allowed a 
designer to implement a mechanism to personalize a document for a particular user, as taught by 
Graham in the Abstract. These references were all applicable to the same field of endeavor, i.e., 
annotation of paper documents with electronic information. 

Regarding claim 2: Klemmer teaches module further comprises content recognition 
software for recognizing features in the media content (See Klemmer page 92 in the top 
paragraph of the right column discussing the creating of an MPEG-2 video from a video source.) 

Regarding claim 3: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses processing logic for controlling a printer driver interface 
associated with the printer. (See Graham col. 6 lines 4-7 discussing the use of Postscript 
printing.) 

Regarding claim 4: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses processing logic for controlling a printer console on the 
printer. (See Graham col. 6 lines 4-7 discussing the use of Postscript printing.) 
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Regarding claim 5: Klemmer teaches wherein the feature extraction module is further 
adapted to generate the media representation in digital format. (See Klemmer page 92 in the 
top paragraph of the right column discussing the creating of an MPEG-2 video from a video 
source.) 

Regarding claim 6: Klemmer teaches wherein the feature extraction module is further * 
adapted to generate the media representation in paper format. (See Klemmer page 92 Fig. 3 
and the top paragraph in the right column teaching a video paper system.) 

Regarding claim 8: Klemmer teaches wherein at least one of the user-selectable 
identifiers comprises a barcode printed on the document displaying the media representation. 
(See Klemmer page 92 Fig. 3 showing the incorporation of barcodes on a video paper system 
document.) 

Regarding claim 10: Klemmer teaches wherein the barcode on the document can be 
scanned to play the associated media content segment on a display device. (See Klemmer page 
89 Abstract and page 92 in the 1 st paragraph under "Hardware" discussing barcode scanning 
and video playback.) 
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Regarding claim 12: Klemmer teaches wherein the graphical representation includes 
audio content displayed as an audio waveform timeline. (See Klemmer page 92 Fig. 3 showing 
barcodes associated with Voices augmented paper transcripts in the context of the bottom 
paragraph in the right column discussing the adding of barcodes to time code metadata 
information.) 

Regarding claim 13: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to user-selected segments of media content. (See Klemmer page 92 Fig. 
3 showing barcode markers associated with Voices augmented paper transcripts in the context of 
the bottom paragraph in the right column discussing the adding of barcodes to time code 
metadata information.) 

Regarding claim 14: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to segments of audio content, the segments being defined by a search 
for particular features within the media content. (See Klemmer page 92 Fig. 3 showing 
barcodes associated with Voices augmented paper transcripts in the context of the bottom 
paragraph in the right column discussing the adding of barcode markers to time code metadata 
information.) 
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Regarding claim 15: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to segments of media content, at least one of the markers having text 
information describing the segment of media content. (See Klemmer page 92 Fig. 3 showing 
barcodes and associated textual passages.) 

Regarding claim 16: Klemmer teaches wherein the timeline includes markers along its 
length that each correspond to a segment of the media content, at least one of the markers 
having timestamp information describing the segment of the media content (See Klemmer 
page 92 Fig. 3 showing barcodes associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 

Regarding claim 17: Klemmer teaches wherein the media representation includes a 
header describing the media content. (See Klemmer page 91 bottom paragraph in the right 
column discussing the placement of metadata in a header.) 

Regarding claim 18: Klemmer teaches wherein the feature extraction module is 
further adapted to generate the media representation is generated according to format 
specifications included in a data structure. (See Klemmer page 92 top paragraph in the right 
column discussing the creation of a paper layout based upon a transcript.) 
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Regarding claim 19: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying a format of a 
graphical representation of the media content. (See Klemmer page 92 top paragraph in the 
right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 20: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying a layout of the media 
representation. (See Klemmer page 92 top paragraph in the right column discussing the creation 
of a paper layout based upon a transcript.) 

Regarding claim 21 : Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying the media content 
markers included in the media representation. (See Klemmer page 92 top paragraph in the 
right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 22: Klemmer teaches wherein the format specifications included in 
the data structure comprise a number of user-definable fields specifying the feature extraction 
techniques applied to the media content. (See Klemmer page 92 in the top paragraph in the 
right column discussing the creation of MPEG-2 and the use of a layout transcript.) 
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Regarding independent claim 25: Klemmer teaches ,4 method for generating a 
representation of time-based the method comprising: extracting features from media content; 
(See Klemmer page 92 in the top paragraph of the right column discussing the creating of an 
MPEG-2 video from a video source.) generating a media representation representing the 
features extracted; (See Klemmer page 92 in the top paragraph of the right column discussing 
the making of corresponding JPEG t\ixivribm\\§) formatting the media representation according 
to a representation specification, the formatting including applying the features extracted to 
the media representation extraction information; (See Klemmer page 92 in the top paragraph of 
the right column discussing the creating of a paper layout from a time stamped transcript.) and 
wherein the formatted media representation includes a graphical representation of a timeline 
and a plurality of user selectable identifiers representing the features extracted from the 
media content for selection by a user to play media content segments of a defined length 
associated with each of the features, wherein the plurality of selectable identifiers are linked to 
locations on the timeline. (See Klemmer page 91 in the last paragraph of the right column 
discussing the adding of time code information to the print format. See also page 92 Fig. 3 
showing barcodes linking chronological sections of a book to A/V data and page 92 in the top 
paragraph of the right column discussing the creating of a paper layout from a time stamped 
transcript.) 

However, Klemmer does not explicitly teach the remaining limitations as claimed. 
Graham, though, discloses printing a document displaying the formatted media representation, 
(See Graham Fig. 4A #408 and col. 5 lines 32-50 teaching the printing of a paper reader's 
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assistant document having an imprinted thumbnail and teaching additional information as being a 
"discussion" of user interests.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Graham for the benefit of Klemmer, because to do so allowed a 
designer to implement a mechanism to personalize a document for a particular user, as taught by 
Graham in the Abstract. These references were all applicable to the same field of endeavor, i.e., 
annotation of paper documents with electronic information. 

Regarding claim 27: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses wherein extracting features of media content further 
comprises performing keyword searching on the media content (See Graham col. 7 lines 50-65 
discussing the use of keywords and keyphrases.) 

Regarding claim 31 : Klemmer teaches wherein the graphical representation includes 
an audio content waveform displayed along a timeline (See Klemmer page 92 Fig 3 showing 
text data displayed, the particular data chosen to display having been an obvious variant. See 
also page 92 Fig. 3 showing barcodes associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 
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Regarding claim 33: Klemmer teaches wherein at least one of the user-selectable 
identifiers comprises a barcode printed on the document displaying media representation. 
(See Klemmer page 92 Fig. 3 showing barcodes printed on a video paper document.) 

Regarding claim 34: Klemmer teaches wherein the user scans the barcode to play the 
associated media content on a display device. (See Klemmer page 92 in the 1 st paragraph in the 
right column under "Hardware" discussing the use of a barcode scanner.) 

Regarding claim 35: Klemmer teaches/wrf/ier comprising generating markers along 
the timeline, the markers corresponding to user-selected media content. (See Klemmer page 
92 Fig. 3 showing barcodes, in the context of page 91 in the bottom paragraph in the right 
column discussing the adding of timecode metadata.) 

Regarding claim 36: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers corresponding to features extracted from the media 
content. (See Klemmer page 92 Fig. 3 showing barcodes associated with Voices augmented 
paper transcripts in the context of the bottom paragraph in the right column discussing the adding 
of barcode markers to time code metadata information.) 

Regarding claim 37: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers including text information describing the media 
content. (See Klemmer page 92 Fig. 3 showing barcodes and associated textual passages.) 
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Regarding claim 38: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers including timestamp information describing the media 
content, (See Klemmer page 92 Fig. 3 showing barcodes associated with Voices augmented 
paper transcripts in the context of the bottom paragraph in the right column discussing the adding 
of barcodes to time code metadata information.) 

Regarding claim 39: Klemmer teaches wherein printing the media representation 
further comprises printing a header describing the media content. (See Klemmer page 91 
bottom paragraph in the right column discussing the placement of metadata in a header.) 

Regarding claim 40: Klemmer teaches wherein printing the media representation 
further comprises generating a representation in digital format, (See Klemmer page 92 in the 
top paragraph of the right column discussing the creating of an MPEG-2 video from a video 
source.) 

Regarding claim 41: Klemmer teaches wherein printing the media representation 
further comprises printing a representation in paper format, (See Klemmer page 92 Fig. 3 
showing "Books with Voices augmented paper transcripts".) 
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Regarding claim 42: Klemmer teaches wherein formatting the media representation 
according to the representation specification further comprises defining a format of the media 
representation using a data structure with format specifications. (See Klemmer page 92 top 
paragraph in the right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 44: Klemmer teaches further comprising applying a barcode generation 
algorithm to render a barcode image including identifier information. (See Klemmer page 92 
Fig. 3 showing barcodes that link to further information, it being implied that such information 
requires an identifier for the purposes of storing/locating that information.) 

Regarding claim 45: Klemmer teaches further comprising applying a barcode 
algorithm to render a barcode image including timestamp information. (See Klemmer page 92 
top paragraph in the right column discussing the creation of a paper layout based upon a 
transcript.) 

8. Claims 28-29 are rejected under 35 U.S.C. 103(a) as being unpatentable over Scott R. 
Klemmer et al. ("Books With Voices: Paper Transcripts as a Tangible Interface to Oral 
Histories", CHI 2003 , Fort Lauderdale, FL, Apr. 5-10, 2003, pp. 89-96, hereafter referred to as 
"Klemmer") in view of Graham et al. (US Patent No. 6,369,81 1, filed Sep. 9, 1998 and issued 
Apr, 9, 2002, hereafter referred to as "Graham") and Ponceleon et al. (US Patent Application 
Publication No. 2003/0187642, filed Mar. 29, 2002 and published Oct. 2, 2003, hereafter 
referred to as "Ponceleon"). 
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Regarding claim 28: Klemmer does not explicitly teach the remaining limitations as 
claimed. Ponceleon, though, discloses wherein extracting features of media content further 
comprises performing speech recognition on the media content (See Ponceleon Abstract and 
paragraph [0003] discussing the use of automatic speech recognition for discovering salient 
sections in a speech transcription.) 

■ It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Ponceleon for the benefit of Klemmer in view of Graham, because to do 
so enabled a designer to implement a system to automatically discover salient segments in a 
speech transcript, as taught by Ponceleon in the Abstract. These references were all applicable to 
the same field of endeavor, i.e., automation information retrieval. 

Regarding claim 29: Klemmer does not explicitly teach the remaining limitations as 
claimed. Ponceleon, though, discloses wherein extracting features of media content further 
comprises performing event detection on the media content. (See Ponceleon paragraph [0007] 
discussing the detection of events occurring in the news.) 
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Conclusion 



9. 



The prior art made of record and not relied upon is considered pertinent to applicant's 



disclosure. 



Non-Patent Literature 

Hecht, David L., "Printed Embedded Data Graphical User Interfaces", Computer . 
Vol. 34, Issue 3, Mar. 200 1 , pp. 47-55. 

Arai, Toshifumi, et al., "PaperLink: A Technique for Hyperlinking from Real Paper 
to Electronic Content", CHI 97 . Atlanta, GA, Mar. 22-27, 1997, pp. 327-334. 



US Patents 



Philyaw et al 
Philyaw et al 



6,745,234 
6,098,106 



Application/Control Number: 

10/814,844 

Art Unit: 2162 



Page 17 



Contact Information 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Robert Stevens whose telephone number is (571) 272-4102. The 
examiner can normally be reached on M-F 6:00 - 2:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John E. Breene can be reached on (571) 272-4107. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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